Efficient Scheduling of Complete Exchange on Clusters

One of the performance limitations of clusters is their message passing capability, while complete exchange is known to be the severest communication pattern on all types of message passing machines. In this paper, we focus on the practical issues of designing high-speed complete exchange algorithms on a commodity cluster interconnected by a non-blocking crossbar switch. Four complete exchange algorithms, including, shift exchange, pairwise exchange, group shuffle exchange and synchronous shuffle exchange algorithms are implemented and tested on a cluster platform. To avoid node and link contention, these algorithms schedule the communication at the packet-level. They aim at fully utilizing the available communication bandwidth both in the links and switch and avoiding the Head-Of-Line problem which would stall the pipelines and decrease the overall efficiency. Both the analytical and measured results show that the synchronous shuffle exchange algorithm can achieve the best performance. It can reach 97% of the available bandwidth in our tests; while the group shuffle exchange performs almost as good as the synchronous shuffle exchange algorithm but scales better when it works on an input-buffered switch. Both shift and pairwise exchanges become inefficient when exchanging small messages, as they cannot fill up those network pipelines effectively. Performance studies of the four algorithms on both input-buffered and shared-buffered switches are also reported.